Deep learning based medical system and method for image acquisition

ABSTRACT

A medical imaging system includes at least one medical imaging device providing image data of a subject and a processing system programmed to generate a plurality of training images having simulated medical conditions by blending a pathology region from a plurality of template source images to a plurality of target images. The processing system is further programmed to train a deep learning network model using the plurality of training images and input the image data of the subject to the deep learning network model. The processing system is further programmed to generate a medical image of the subject based on the output of the deep learning network model.

BACKGROUND

The field of the disclosure relates generally to medical imaging systemsand methods, and more particularly, to techniques of generatingsimulated images with patient conditions/pathology for machinelearning-based applications.

In modern healthcare facilities, non-invasive medical imaging systemsare often used for identifying, diagnosing, and treating physicalconditions. Medical imaging encompasses different non-invasivetechniques used to image and visualize the internal structures and/orfunctional behavior (such as chemical or metabolic activity) of organsand tissues within a patient. Currently, a number of modalities ofmedical diagnostic and imaging systems exist, each typically operatingon different physical principles to generate different types of imagesand information. These modalities include ultrasound systems, computedtomography (CT) systems, X-ray systems (including both conventional anddigital or digitized imaging systems), positron emission tomography(PET) systems, single photon emission computed tomography (SPECT)systems, and magnetic resonance (MR) imaging systems.

Similar to other technology domains, Deep leaming (DL) has made asignificant foray into the medical imaging domain as well. DL is beingused in many of the imaging modalities including, CT, PET, X-ray, SPECTand MR imaging systems. Deep learning uses efficient techniques to makethe diagnosis in state of the art manner. In general, DL is a subset ofmachine learning where artificial neural network models learn from largeamounts of training data. Applications of deep learning include medicalimage preprocessing (i.e., de-noising and enhancement), medical imagesegmentation, and medical image object detection and recognition.

In DL based medical imaging, the expectation is for the DL network towork robustly across a range of patient conditions such as pathologiesof varying degrees, implants etc. The location and degree of impact ofsuch patient conditions on the medical images can be dramatic andpotentially affect the performance of DL techniques. For example, in amagnetic resonance imaging (MRI) system, the metal implant can destroythe signal in the vicinity of the tissue and obscure pertinentlandmarks. Another example, could include, fracture condition, whichdistorts the bone structure or a topology of a pathology, e.g. BIRADSclassification requiring certain shapes for classification.

Therefore, there is a need for an improved magnetic resonance imagingsystem and method.

BRIEF DESCRIPTION

In accordance with an embodiment of the present technique, a medicalimaging system is provided. The system includes at least one medicalimaging device providing image data of a subject and a processing systemprogrammed to generate a plurality of training images having simulatedmedical conditions. The processing system is further programmed to traina deep learning network model using the plurality of training images andinput the image data of the subject to the deep learning network model.The processing system is further programmed to generate a medical imageof the subject based on the output of the deep learning network model.The processing system generates the plurality of training images byblending a pathology region from a plurality of template source imagesto a plurality of target images.

In accordance with another embodiment of the present technique, a methodfor imaging a subject is presented. The method includes generating imagedata of the subject with a medical imaging device and generating aplurality of training images having simulated medical conditions byblending a pathology region from a plurality of template source imagesto a plurality of target images. The method further includes training adeep learning network model using the plurality of training images andproviding the image data of the subject as an input to the deep learningnetwork model. The method also includes generating a medical image ofthe subject based on the output of the deep learning network model.

DRAWINGS

These and other features, aspects, and advantages of the presentinvention will become better understood when the following detaileddescription is read with reference to the accompanying drawings in whichlike characters represent like parts throughout the drawings, wherein:

FIG. 1 is a schematic diagram of an exemplary magnetic resonance imaging(MRI) system in accordance with an embodiment of the present technique;

FIG. 2A is an exemplary truncation artifact reduction/classifier systemwhich is used in the MRI system of FIG. 1;

FIG. 2B is a flow chart of an exemplary method that may be implementedin system of FIG. 2A in accordance with an embodiment of the presenttechnique;

FIG. 3 is a schematic diagram of an example DL network that may be usedin the truncation artifact reduction/classifier system of FIG. 2A;

FIG. 4 is a schematic diagram of a method for generating training imagesfor the DL network of FIG. 3 in accordance with another embodiment ofthe present technique;

FIGS. 5A, 5B and 5C are exemplary simulated training images generated inaccordance with an embodiment of the present technique;

FIG. 6 is a schematic diagram depicting experimental results of a deeplearning network in accordance with an embodiment of the presenttechnique;

FIG. 7 is a schematic diagram depicting medical images generated by theMRI system of FIG. 1 in accordance with an embodiment of the presenttechnique; and

FIG. 8 is a flowchart depicting a method for imaging a subject inaccordance with an embodiment of the present technique.

DETAILED DESCRIPTION

One or more specific embodiments will be described below. In an effortto provide a concise description of these embodiments, all features ofan actual implementation may not be described in the specification. Itshould be appreciated that in the development of any such actualimplementation, as in any engineering or design project, numerousimplementation-specific decisions must be made to achieve thedevelopers' specific goals, such as compliance with system-related andbusiness-related constraints, which may vary from one implementation toanother. Moreover, it should be appreciated that such a developmenteffort might be complex and time consuming, but would nevertheless be aroutine undertaking of design, fabrication, and manufacture for those ofordinary skill having the benefit of this disclosure.

When introducing elements of various embodiments of the presentembodiments, the articles “a,” “an,” “the,” and “said” are intended tomean that there are one or more of the elements. The terms “comprising,”“including,” and “having” are intended to be inclusive and mean thatthere may be additional elements other than the listed elements.Furthermore, any numerical examples in the following discussion areintended to be non-limiting, and thus additional numerical values,ranges, and percentages are within the scope of the disclosedembodiments. Furthermore, the terms “circuit” and “circuitry” and“controller” may include either a single component or a plurality ofcomponents, which are either active and/or passive and are connected orotherwise coupled together to provide the described function.

The presented technique includes systems and methods of removingartifacts in medical images or classifying images in the presence ofartifacts using a deep learning model. As used herein, a subject is ahuman (or patient), an animal, or a phantom. Unlike signals, whichrepresent the anatomies or structures of the subject, artifacts arevisual anomalies in the medical images that are not present in thesubject, but may be caused by subject conditions such as metal implantsor fractures in a human body. Method aspects will be in part apparentand in part explicitly discussed in the following description.

In general, the location and degree of impact of patient conditions(e.g., metal implants or fractures) on the medical images can bedramatic and potentially affect the performance of DL techniques. Oneway to mitigate this problem is to gather such relevant patient data toexpose the training of the DL network to learn the intended task despitethe presence of such patient conditions. Some of these conditions occurvery rarely thus limiting the number of examples that can be used totrain the networks.

Another approach would be to use bio-physical models which may notexactly replicate the patient conditions besides being computationallyintensive. Deep-learning based image synthesis approaches do exist.However, their applicability is marred by the inflexibility to controlthe contrast, location, intensity variations etc. in synthetic data;besides requiring large amounts of data to generate the model.

The embodiments presented herein are related to techniques of generatingtraining images having simulated patient conditions for DL-basedapplications. It should also be noted that although, the techniqueherein is presented with respect to a magnetic resonance imaging (MRI)system, the present technique may also be employed in other imagingsystems that use DL algorithms. Further, although, the technique hereinis presented in detail with respect to metal implants, the technique mayalso be used for other medical conditions such as a bone fracture.

FIG. 1 illustrates a schematic diagram of an exemplary MRI system 10. Inthe exemplary embodiment, the MRI system 10 includes a workstation 12having a display 14 and a keyboard 16. The workstation 12 includes aprocessor 18, such as a commercially available programmable machinerunning a commercially available operating system. The workstation 12provides an operator interface that allows scan prescriptions to beentered into the MRI system 10. The workstation 12 is coupled to a pulsesequence server 20, a data acquisition server 22, a data processingserver 24, and a data store server 26. The workstation 12 and eachserver 20, 22, 24, and 26 communicate with each other.

In the exemplary embodiment, the pulse sequence server 20 responds toinstructions downloaded from the workstation 12 to operate a gradientsystem 28 and a radiofrequency (“RF”) system 30. The instructions areused to produce gradient and RF waveforms in MR pulse sequences. An RFcoil 38 and a gradient coil assembly 32 are used to perform theprescribed MR pulse sequence. The RF coil 38 is shown as a whole body RFcoil. The RF coil 38 may also be a local coil that may be placed inproximity to the anatomy to be imaged, or a coil array that includes aplurality of coils.

In the exemplary embodiment, gradient waveforms used to perform theprescribed scan are produced and applied to the gradient system 28,which excites gradient coils in the gradient coil assembly 32 to producethe magnetic field gradients G_(x), G_(y), and G_(z), used forposition-encoding MR signals. The gradient coil assembly 32 forms partof a magnet assembly 34 that also includes a polarizing magnet 36 andthe RF coil 38.

In the exemplary embodiment, the RF system 30 includes an RF transmitterfor producing RF pulses used in MR pulse sequences. The RF transmitteris responsive to the scan prescription and direction from the pulsesequence server 20 to produce RF pulses of a desired frequency, phase,and pulse amplitude waveform. The generated RF pulses may be applied tothe RF coil 38 by the RF system 30. Responsive MR signals detected bythe RF coil 38 are received by the RF system 30, amplified, demodulated,filtered, and digitized under direction of commands produced by thepulse sequence server 20. The RF coil 38 is described as a transmitterand receiver coil such that the RF coil 38 transmits RF pulses anddetects MR signals. In one embodiment, the MRI system 10 may include atransmitter RF coil that transmits RF pulses and a separate receivercoil that detects MR signals. A transmission channel of the RF system 30may be connected to a RF transmission coil and a receiver channel may beconnected to a separate RF receiver coil. Often, the transmissionchannel is connected to the whole body RF coil 38 and each receiversection is connected to a separate local RF coil.

In the exemplary embodiment, the RF system 30 also includes one or moreRF receiver channels. Each RF receiver channel includes an RF amplifierthat amplifies the MR signal received by the RF coil 38 to which thechannel is connected, and a detector that detects and digitizes the Iand Q quadrature components of the received MR signal. The magnitude ofthe received MR signal may then be determined as the square root of thesum of the squares of the I and Q components as in Eq. (1) below:

M=√{square root over (I ² +Q ²)}  (1);

and the phase of the received MR signal may also be determined as in Eq.(2) below:

$\begin{matrix}{\varphi = {{\tan^{- 1}\left( \frac{Q}{I} \right)}.}} & (2)\end{matrix}$

In the exemplary embodiment, the digitized MR signal samples produced bythe RF system 30 are received by the data acquisition server 22. Thedata acquisition server 22 may operate in response to instructionsdownloaded from the workstation 12 to receive real-time MR data andprovide buffer storage such that no data is lost by data overrun. Insome scans, the data acquisition server 22 does little more than passthe acquired MR data to the data processing server 24. In scans thatneed information derived from acquired MR data to control furtherperformance of the scan, however, the data acquisition server 22 isprogrammed to produce the needed information and convey it to the pulsesequence server 20. For example, during prescans, MR data is acquiredand used to calibrate the pulse sequence performed by the pulse sequenceserver 20. Also, navigator signals may be acquired during a scan andused to adjust the operating parameters of the RF system 30 or thegradient system 28, or to control the view order in which k-space issampled.

In the exemplary embodiment, the data processing server 24 receives MRdata from the data acquisition server 22 and processes it in accordancewith instructions downloaded from the workstation 12. Such processingmay include, for example,

Fourier transformation of raw k-space MR data to produce two orthree-dimensional images, the application of filters to a reconstructedimage, the performance of a backprojection image reconstruction ofacquired MR data, removal of artifacts in the MR data, classification ofMR images in presence of artifacts, the generation of functional MRimages, and the calculation of motion or flow images.

In the exemplary embodiment, images reconstructed by the data processingserver 24 are conveyed back to, and stored at, the workstation 12. Insome embodiments, real-time images are stored in a database memory cache(not shown in FIG. 1), from which they may be output to operator display14 or a display 46 that is located near the magnet assembly 34 for useby attending physicians. Batch mode images or selected real time imagesmay be stored in a host database on disc storage 48 or on a cloud. Whensuch images have been reconstructed and transferred to storage, the dataprocessing server 24 notifies the data store server 26. The workstation12 may be used by an operator to archive the images, produce films, orsend the images via a network to other facilities.

As discussed earlier, the acquired MR data from the data acquisitionserver 22 may include artifacts due to pathology condition of thesubject such as metal implants or fractures in a human body. Using deeplearning to directly remove these artifacts provides superiorperformance to conventional methods.

FIG. 2A is a schematic diagram of an exemplary artifactreduction/classifier system 200 which is used in MRI system 10 ofFIG. 1. In the exemplary embodiment, the system 200 includes a computingdevice 202 which is configured to classify images or reduce artifactstherein. The computing device 202 includes a DL network model 204. Thesystem 200 also includes a training images generator 206 to generatetraining images in accordance with an embodiment of the presenttechnique. The training images from the training images generator 206are used to train the DL network model 204. In one embodiment of thepresent technique, a training images generator is presented to generatethese training images having simulated medical conditions. In oneembodiment, the training images are images having simulated artifactswhich are result of a medical condition of the patient. The computingdevice 202 may then use the trained DL network model 204 to generatemedical images with classification as having a certain medical conditionor medical images with reduced artifact. The computing device 202 may beincluded in the workstation 12 of the MRI system 10, or may be includedon a separate computing device that is in communication with theworkstation 12. Further, in another embodiment, training imagesgenerator 206 may be included on a separate computing device and maygenerate the training images well ahead of the actual operation of theMRI system 10.

FIG. 2B is a flow chart of an exemplary method 250. The method 250 maybe implemented on the artifact reduction/classifier system 200. In theexemplary embodiment, the method includes acquiring 252 a plurality oftemplate source images and a plurality of target images. The pluralityof template source images includes representative examples of medicalconditions of the patient. For example, the plurality of template sourceimages may include images of the patient having metal implants in kneeor other body parts. The plurality of template source images may alsoinclude images with fractured bones. Further, the plurality of targetimages is selected from a set of patient images having no medicalconditions. Both, the template source images and target images may bestored in a historical image database of the same patient or of variouspathologies of different patients from which these images may beacquired when required.

The method 250 further includes deriving 254 segmentation masks of apathology region from the plurality of template source images. Themethod 250 also includes blending 256 segmentation masks to theplurality of target images to generate a plurality of training imageshaving medical conditions. Finally, the DL network model is executed atstep 258 to generate a medical image of the patient having reducedartifacts or medical image with classification as having a certainmedical condition. In one embodiment, the method steps 252, 254 and 256may be implemented in training images generator 206 of FIG. 2A.

FIG. 3 is a representation of an example DL network model 300 that maybe used as the DL network model 204 in the embodiment of 200. Theexample DL network model 300 includes layers 320, 340, 360, and 380. Thelayers 320 and 340 are connected with neural connections 330. The layers340 and 360 are connected with neural connections 350. The layers 360and 380 are connected with neural connections 370. Data flows forwardvia inputs 312, 314, 316 from the input layer 320 to the output layer380 and to an output 390. The inputs 312, 314, 316 may be source imagesand output 390 may be a target images.

The layer 320 is an input layer that, in the example of FIG. 3, includesa plurality of nodes 322, 324, 326. The layers 340 and 360 are hiddenlayers and include, the example of FIG. 3, nodes 342, 344, 346, 348,362, 364, 366, 368. The DL network model 300 may include more or lesshidden layers 340 and 360 than shown. The layer 380 is an output layerand includes, in the example of FIG. 3 a node 382 with an output 390.Each input 312-316 corresponds to a node 322-326 of the input layer 320,and each node 322-326 of the input layer 320 has a connection 330 toeach node 342-348 of the hidden layer 340. Each node 342-348 of thehidden layer 340 has a connection 350 to each node 362-368 of the hiddenlayer 360. Each node 362-368 of the hidden layer 360 has a connection370 to the output layer 380. The output layer 380 has an output 390 toprovide an output from the example DL network model 300.

Of connections 330, 350, and 370 certain example connections 332, 352,372 may be given added weight while other example connections 334, 354,374 may be given less weight in the DL network model 300. Input nodes322-326 are activated through receipt of input data via inputs 312-316,for example. Nodes 342-348 and 362-368 of hidden layers 340 and 360 areactivated through the forward flow of data through the network model 300via the connections 330 and 350, respectively. Node 382 of the outputlayer 380 is activated after data processed in hidden layers 340 and 360is sent via connections 370. When the output node 382 of the outputlayer 380 is activated, the node 382 outputs an appropriate value basedon processing accomplished in hidden layers 340 and 360 of the DLnetwork model 300.

FIG. 4 is a schematic diagram depicting a method 400 for generatingtraining images for the DL network model in accordance with anotherembodiment of the present technique. The method 400 may be implementedin training images generator 206 of FIG. 2A. The method 400 includesobtaining a template metal volume at step 402. The template metal volumecomprises template source images that include representative examples ofmetal implants in a patient knee as shown in knee images 404. Thetemplate metal volume 402 is obtained from a database of historicalimages of patients having metal implants such as screws located in theirknees, for example. At step 406, the method includes segmenting metalregions (i.e., segmentation masks) from the template metal volume 402.For example, based on the geometry of the screw, a segmentationalgorithm such as a ‘connected pixel algorithm’ may be used to segmentor detect the metal region in the knee. In general, the segmentationalgorithm determines which pixels of the images 404 belong to whichobjects.

At step 408, a target normal volume is obtained. The target normalvolume 408 comprises target images that include representative examplesof the patient knee having no medical conditions as shown in a kneeimage 410. Step 412 includes processing segmented metal regions. In oneembodiment, processing of segmented metal region includes imageregistration, landmark matching, histogram matching, or combinationsthereof between the segmented metal regions and target normal volume.The image registration involves transforming different sets of imagesinto one coordinate system. In general, the segmented metal region maybe from template source images which are of different people (adults orchildren) i.e., different knee sizes. Thus, image registration is usedfor scale matching to accommodate for the differences in sizes oftemplate source images and the target images.

Further, the segmented metal regions may be at different locations inthe patient body. Therefore, in one embodiment, landmark matching ofsegmented metal regions is used to align the segmented metal regionswith the target images. As will be appreciated by those skilled in theart, a small deformation image matching or a large deformation imagematching algorithm may be used for landmark matching. Finally, histogrammatching may be used to normalize the segmented metal regions tocompensate for variations in imaging system sensors, atmosphericconditions or intensity variations. As will be appreciated by thoseskilled in the art, histogram matching includes transformation ofsegmented metal regions to match their histogram with that of targetimages.

The method 400 further includes augmentations of metal regions atlandmark points or regions of interest in the target images at step 414.In one embodiment, the region of interest in the target images may bedetermined using attention maps derived from another machine learningnetwork or based on regions most likely to have the medical conditione.g., surgical implants. In another embodiment, region of interest inthe target images may be determined using ground truth marking on theplurality of target images or using an atlas based method. Further, theaugmentation process includes a series of transformations of segmentedmetal regions (i.e., segmentation masks). The series of transformationsinclude rotation, resizing and elastic deformation of the segmentedmetal regions. In general, there are not many template source imagesavailable having medical conditions. For example, if there are 100template source images then out of 100 only 5 or 10 may have somemedical conditions such as metal implant or fracture. Further, theaugmentation process may include pasting the segmentation masks from oneregion in the plurality of template source images to a different regionin the plurality of target images. In other words, the differentlocations of metal implants may be simulated in one embodiment.Therefore, the augmentation step is performed to expand training imagesfor the deep learning network model by simulating various patientconditions with variations in segmentation masks. For example, the kneescrews may be augmented at 5-6 landmark points in the target image orthe different sizes of screws may be augmented in the target image.Thus, from a single template source image having a medical condition itis possible to generate 20 to 30 training images with simulated medicalconditions. Finally, at step 416, segmented metal regions are blendedwith the target images. The blending operation ensures seamless blendingor pasting of augmented metal regions with the target images as shown inimages 418. The blending process may include performing a contrastequalization of the plurality of template source images and theplurality of target images for obtaining perceptual smoothness betweenthe plurality of target images and the pasted segmented masks.

FIGS. 5A-5C are exemplary simulated training images generated inaccordance with an embodiment of the present technique. FIG. 5A showsaxial sample 500 of knee images. Specifically, image 502 corresponds toblending of a metal region scaled to completely obscure the femoralcondyle landmark. Image 504 shows the metal region placed away from thelandmark. Image 506 shows randomly placed metal and image 508 shows theblending of the metal above Tibia Landmark.

FIG. 5B shows coronal sample 510 of knee images. In FIG. 5B, image 512corresponds to a randomly placed metal and image 514 corresponds to themetal placed below meniscus. Further, image 516 shows the metal placedon meniscus with aspect ratio of the metal being preserved. Finally,image 518 shows the metal placed above meniscus.

FIG. 5C shows sagittal sample 520 of knee images. In FIG. 5C, image 522shows a randomly placed metal in the knee. Image 524 corresponds to themetal placed below meniscus and image 526 shows the metal placed onmeniscus and made to scale the entire region of interest. Further, image528 shows the metal placed above meniscus.

FIG. 6 shows a schematic diagram 600 depicting experimental results of adeep learning network model in accordance with an embodiment of thepresent technique. The experiment was conducted on test image samplescorrupted by metal implants in actual clinical subjects. In FIG. 6,plots 602 and 604 show results of a deep learning network model trainedwithout the simulated training images and trained with the simulatedtraining images as described above. Horizontal axis 606 in both plots602 and 604 shows true or real labels of test samples whereas verticalaxis 608 shows predicted labels by the deep learning network model ofthe test samples.

In plot 602, out of total 226 test samples of Femur images, the networkclassified (or predicted) 210 images accurately as Femur images whereasremaining 16 images were incorrectly classified as either Tibia images(9) or as noise (7). In comparison, plot 604 classified 211 imagesaccurately as Femur images. Further, out of total 105 test samples ofTibia images, the network in plot 602 classified 59 images accurately asTibia images whereas the network in plot 604 classified 71 imagesaccurately as Tibia images. Moreover, out of total 466 test samples ofirrelevant or noise images, the network in plot 602 classified 294images accurately as noise images whereas the network in plot 604classified 389 images accurately as Tibia images. Similarly, out oftotal 225 test samples of coronal images, the network in plot 602classified 209 images accurately as coronal images whereas the networkin plot 604 classified 210 images accurately as coronal images. Finally,out of total 277 test samples of Sagittal images, the network in plot602 classified 263 images accurately as Sagittal images whereas thenetwork in plot 604 classified 268 images accurately as Sagittal images.

Overall, the experiment showed that the DL network model trained withoutthe simulated training images i.e., plot 602 had classified 1035 imagesaccurately out of total 1299 images i.e., accuracy of 79.67%. Incomparison, the DL network model trained with simulated training imagesi.e., plot 604 had classified 1149 images accurately out of total 1299images i.e., accuracy of 88.45%. Thus, there was improvement in accuracyof about 9% with the DL network model trained with the simulatedtraining images in accordance with the embodiment of the presenttechnique.

FIG. 7 shows a schematic diagram 700 depicting medical images generatedby an MRI system (e.g., System 10 of FIG. 1) in accordance with anembodiment of the present technique. In general, FIG. 7 shows medicalimages 702, 704, 706, 708, 710 and 712 corresponding to SagittalCoverage mask, Mensicus plane Sagittal, Mensicus plane coronal, FemoralCondyle Inner Lateral Plane (FCIL), Tibia plane and Femoral CoronalPlane (FCP) respectively. It can be seen from these medical images, thateven in the presence of metal artifacts, the present technique is ableto show the landmarks (horizontal lines) quite accurately. It should benoted that in images 702 and 710, the arrows point to the metal region(i.e., white halo), whereas in image 708, the arrow points to the ghostregion (i.e., dark portion).

FIG. 8 shows a flowchart depicting a method 800 for imaging a subject inaccordance with an embodiment of the present technique. At step 802, themethod includes generating image data of a subject with a medicalimaging device. In one embodiment, the medical imaging device includesan MRI system. The image data generated by the MRI system may includeartifacts due to medical conditions present in the subject body.Therefore, the method includes processing the image data to reduce theartifacts or to classify these images even in presence of theseartifacts.

At step 804, the method includes generating a plurality of trainingimages having simulated medical conditions by blending a pathologyregion from a plurality of template source images to a plurality oftarget images. The template source images may be obtained from adatabase of historical images of patients having metal implants such asscrews located in their knees or bone fractures etc. Further, the targetimages are selected from a set of patient images having no medicalconditions. Moreover, in method 800, blending the pathology regioncomprises deriving segmentation masks of the pathology region from theplurality of template source images and processing the segmentationmasks. In one embodiment, processing the segmentation masks includesimage registration, landmark matching, histogram matching, orcombinations thereof between the segmentation masks and target images.Further, the blending includes an augmentation process where thesegmentation masks are augmented at landmark points or regions ofinterest in target images. The augmentation process includestransforming the segmentation masks undergo a series of transformationsbefore being pasted on the region of interest of the plurality of targetimages. In one embodiment, the region of interest of the plurality oftarget images may be determined using attention maps derived fromanother machine learning network or based on regions most likely to havethe medical condition e.g., surgical implants. In another embodiment,region of interest of the plurality of target images may be determinedusing ground truth marking on the plurality of target images or using anatlas based method. The series of transformations include rotation,resizing and elastic deformation of the segmented metal regions.

The method 800 further includes training a deep learning network modelusing the plurality of training images at step 806. The image data fromstep 802 is provided as an input to the trained deep learning networkmodel at step 808. Finally, at step 810, a medical image of the subjectis generated based on the output of the deep learning network model.

The advantages of the present technique include providing flexibility tothe user to synthesize the patient condition at locations of interest asdriven by the task or the deep learning features guiding the task. Thetechnique also overcomes the computational complexity of synthesizingsuch data using bio-physical models(susceptibility simulation in MR formetal etc.) or data complexity (various patient conditions andcorrespondences) and inflexibility (image intensity, blending ratioetc.) for deep learning based synthesis.

This written description uses examples to disclose the invention,including the best mode, and also to enable any person skilled in theart to practice the invention, including making and using any devices orsystems and performing any incorporated methods. The patentable scope ofthe invention is defined by the claims, and may include other examplesthat occur to those skilled in the art. Such other examples are intendedto be within the scope of the claims if they have structural elementsthat do not differ from the literal language of the claims, or if theyinclude equivalent structural elements with insubstantial differencesfrom the literal languages of the claims.

1. A medical imaging system comprising: at least one medical imagingdevice providing image data of a subject; a processing system programmedto: generate a plurality of training images having simulated medicalconditions; train a deep learning network model using the plurality oftraining images; input the image data of the subject to the deeplearning network model; and generate a medical image of the subjectbased on the output of the deep learning network model; wherein theprocessing system is programmed to generate the plurality of trainingimages by blending a pathology region from a plurality of templatesource images to a plurality of target images.
 2. The medical imagingsystem of claim 1, wherein the plurality of template source imagesincludes representative examples of medical conditions of the patientand wherein the plurality of target images is selected from a set ofimages having no medical conditions.
 3. The medical imaging system ofclaim 2, wherein the processing system is programmed to determine thepathology region from the plurality of template source images byderiving segmentation masks of the pathology region from the pluralityof template source images.
 4. The medical imaging system of claim 3,wherein the processing system is programmed to blend the pathologyregion by processing the segmentation masks, wherein the processingincludes image registration, landmark matching, histogram matching, orcombinations thereof between the segmentation masks and the plurality oftarget images.
 5. The medical imaging system of claim 4, wherein theprocessing system is programmed to blend the pathology region by pastingthe processed segmentation masks on a region of interest of theplurality of target images.
 6. The medical imaging system of claim 5,wherein the processing system is programmed to determine the region ofinterest based on ground truth marking on the plurality of target imagesor using an atlas based method.
 7. The medical imaging system of claim5, wherein the processing system is programmed to determine the regionof interest using attention maps derived from another machine learningnetwork or based on regions most likely to medical conditions.
 8. Themedical imaging system of claim 5, wherein the segmentation masksundergo a series of transformations before being pasted on the region ofinterest of the plurality of target images.
 9. The medical imagingsystem of claim 8, wherein the series of transformations includerotation, resizing and elastic deformation of the segmentation masks.10. The medical imaging system of claim 5, wherein the processing systemis programmed to paste the segmentation masks from one region in theplurality of template source images to a different region in theplurality of target images.
 11. The medical imaging system of claim 1,wherein the processing system is programmed to perform a contrastequalization of the plurality of template source images and theplurality of target images for obtaining perceptual smoothness betweenthe plurality of target images and the pasted segmented masks.
 12. Themedical imaging system of claim 1, wherein the medical conditionsinclude metal implants or fracture related condition.
 13. A method forimaging a subject comprising: generating image data of the subject witha medical imaging device; generating a plurality of training imageshaving simulated medical conditions by blending a pathology region froma plurality of template source images to a plurality of target images;training a deep learning network model using the plurality of trainingimages; providing the image data of the subject as an input to the deeplearning network model; generating a medical image of the subject basedon the output of the deep learning network model.
 14. The method ofclaim 13, wherein the plurality of target images is selected from a setof images having no medical conditions and wherein the plurality oftemplate source images includes representative examples of the medicalconditions of the subject.
 15. The method of claim 14, whereindetermining the pathology region from the plurality of template sourceimages comprises deriving segmentation masks of the pathology regionfrom the plurality of template source images.
 16. The method of claim15, wherein the blending the pathology region comprises processing thesegmentation masks, wherein the processing includes image registration,landmark matching, histogram matching, or combinations thereof betweenthe segmentation masks and the plurality of target images.
 17. Themethod of claim 15, wherein blending the pathology region comprisespasting the segmentation masks on a region of interest of the pluralityof target images.
 18. The method of claim 18, wherein the region ofinterest is determined based on ground truth marking on the plurality oftarget images or using an atlas based method.
 19. The method of claim17, wherein the segmentation masks undergo a series of transformationsbefore being pasted on the region of interest of the plurality of targetimages.
 20. The method of claim 19, wherein the series oftransformations include rotation, resizing and elastic deformation ofthe segmentation masks.